Data Validation for MEData

medata here is the Mediterranean dataframe with the following changes:

• data.origin: “Sala - PEW” to “sala_pew”
• There are some latitudes that equal integers (1,2,3…)
• protection: 35460 replacements for YES in 1; 16278 replacements for NO in 0; 637 replacements of #N/A in NA.

Some more work needed:

• sort out caps and spaces: site names; spp names (also parentheses); season;
• species names need to be all changed to genus.spp now some with caps and some with spaces
• spp length in cm I guess?
• should we delete the first column?

General overview

medata %>%
    skimr::skim()

## Warning in .x(x): Variable contains value(s) of "" that have been converted
## to "empty".

## Skim summary statistics
##  n obs: 43791 
##  n variables: 22 
## 
## -- Variable type:factor ----------------------------------------------------------------------------------------------------------
##     variable missing complete     n n_unique
##      country   28202    15589 43791        4
##  data.origin       0    43791 43791        6
##   protection       0    43791 43791        3
##       season       0    43791 43791        4
##         site       0    43791 43791       86
##      species       0    43791 43791       92
##                                    top_counts ordered
##    NA: 28202, Isr: 6143, Fra: 3547, Cro: 3155   FALSE
##  Bel: 15589, cla: 10875, Sal: 8402, azz: 3425   FALSE
##        YES: 30161, NO: 13020, #N/: 610, NA: 0   FALSE
##   Sum: 18905, Aut: 16082, Spr: 8693, Win: 111   FALSE
##    ASI: 2516, Cap: 2156, Gdo: 2007, Ach: 1786   FALSE
##    Cor: 7819, Dip: 4488, Dip: 3879, Sym: 3412   FALSE
## 
## -- Variable type:integer ---------------------------------------------------------------------------------------------------------
##        variable missing complete     n    mean     sd   p0  p25  p50  p75
##  age.reserve.yr    7097    36694 43791   30.25  19.18    1   11   40   40
##     enforcement    1480    42311 43791    1.47   1.11    0    1    1    2
##            sp.n       0    43791 43791   15.32 137.41    0    1    1    4
##           trans       0    43791 43791 1222.42 737.48    1  607 1125 1935
##     yr.creation    7097    36694 43791 1982.83  16.05 1960 1974 1974 2002
##   p100     hist
##     57 <U+2585><U+2585><U+2581><U+2581><U+2582><U+2587><U+2581><U+2585>
##      3 <U+2587><U+2581><U+2587><U+2581><U+2581><U+2587><U+2581><U+2587>
##  10000 <U+2587><U+2581><U+2581><U+2581><U+2581><U+2581><U+2581><U+2581>
##   2393 <U+2583><U+2586><U+2587><U+2583><U+2585><U+2583><U+2583><U+2587>
##   2008 <U+2585><U+2581><U+2587><U+2581><U+2582><U+2581><U+2586><U+2583>
## 
## -- Variable type:numeric ---------------------------------------------------------------------------------------------------------
##      variable missing complete     n     mean         sd       p0      p25
##             a       0    43791 43791    0.015     0.0076  0.00046   0.0087
##             b       0    43791 43791    3         0.13    2.61      2.93  
##         depth    6953    36838 43791    9.38      4.03    1.5       7.6   
##           lat     996    42795 43791   36.48      9.95    1        35.16  
##           lon     995    42796 43791   13.6      11.82    1.16      3.17  
##           rug   35590     8201 43791    1.41      0.26    1         1.22  
##   size.notake   16169    27622 43791 2259.86   4756.22   18        65     
##     sp.length    2411    41380 43791   12.74      8.44    0         8     
##          tmax    2540    41251 43791   25.22      2.42   22.07     22.08  
##          tmin    2540    41251 43791   13.93      3.25    9        11.43  
##  total.mpa.ha   12622    31169 43791 4670.72  19543.47   84.4     650     
##      p50      p75       p100     hist
##    0.014    0.018      0.048 <U+2581><U+2585><U+2587><U+2581><U+2581><U+2581><U+2581><U+2581>
##    3.05     3.09       3.39  <U+2581><U+2582><U+2582><U+2582><U+2587><U+2582><U+2581><U+2581>
##    9.1     10         30     <U+2582><U+2586><U+2587><U+2582><U+2581><U+2581><U+2581><U+2581>
##   40.04    42.46      44.94  <U+2581><U+2581><U+2581><U+2581><U+2581><U+2582><U+2583><U+2587>
##    9.08    18.51      35.08  <U+2587><U+2582><U+2582><U+2582><U+2581><U+2581><U+2581><U+2583>
##    1.37     1.57       2.39  <U+2585><U+2587><U+2586><U+2585><U+2582><U+2581><U+2581><U+2581>
##   84.4   1000      15000     <U+2587><U+2581><U+2581><U+2581><U+2581><U+2581><U+2581><U+2581>
##   12       16        150     <U+2587><U+2582><U+2581><U+2581><U+2581><U+2581><U+2581><U+2581>
##   25.03    27.73      29     <U+2587><U+2581><U+2583><U+2583><U+2585><U+2581><U+2582><U+2586>
##   13       15.92      25     <U+2581><U+2587><U+2583><U+2583><U+2581><U+2581><U+2581><U+2581>
##  650     2375     207000     <U+2587><U+2581><U+2581><U+2581><U+2581><U+2581><U+2581><U+2581>

• Country can be completed from data origin? I need a list of what each one means
• Protection: fix #N/A
• Spcecies: I would like to group into local and non-local (for comparison later)
• Lat: South ~30.276670; North ~45.137447 (should be) we have some that are too low (in the following)
• Lon: West ~ -5.587321; East ~35.110782
• what’s A and B?

summary(medata$lat)

##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
##    1.00   35.16   40.04   36.48   42.46   44.94     996

medata %>% 
  filter(lat < 30) %>% 
  write.csv(file = "lat_probs.csv")

Location and Max Temp

p1 <- ggplot(medata, aes(x = medata$lon, y = medata$lat)) +
  geom_jitter(aes(colour = medata$tmax), show.legend = T) +
  xlab("Longitude") + ylab("Latitude") +
  scale_color_gradient(name = "Max Annual Temp", low = "#3c9ab1", high = "#f22300", na.value = "#899da4")

ggplotly(p1)